A Low-Rank Approximation for MDPs via Moment Coupling

Markov Decision Process Tayloring for Approximation Design Optimal control problems are difficult to solve on large state spaces, calling the development of approximate solution methods. In “A Low-rank MDPs via Moment Coupling,” Zhang and Gurvich introduce a novel framework decision processes (MDPs) that stands two pillars: (i) aggregation, as algorithmic infrastructure, (ii) central-limit-theorem-type approximations, mathematical underpinning. The theoretical guarantees grounded in approximation Bellman equation by partial differential (PDE) where, spirit central limit theorem, transition matrix controlled chain is reduced its local first second moments. Instead solving PDE, algorithm introduced paper constructs “sister”' (controlled) whose moments approximately identical with those focal chain. Because this moment matching, original sister coupled through facilitating optimality guarantees. Embedded into standard soft matching provides disciplined mechanism tune aggregation disaggregation probabilities.